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ABSTRACT 


Neural  net  models  and  their  analogs  present  a  new  approach  to  signal 
processing  that  is  collective,  robust,  and  fault  tolerant.  The  collective 
nature  of  processing  means  also  high  throughput  where  bandwidth  or  speed  of 
the  processing  elements  i.e.,  temporal  degrees  of  freedom  are  traded  by 
spatial  degrees  of  freedom  via  massive  interconnectivity  of  the  elements 
(neurons)  in  the  network.  As  a  result  neural  nets  can  solve  computationally 
extensive  problems  like  those  encountered  for  example  in  optimization, 
nearest  neighbor  search,  inverse  scattering  in  a  matter  of  a  few  time- 
constants  of  the  processing  elements.  In  biological  systems  (the  brain  in 
particular)  this  is  of  the  order  of  a  tens  of  milliseconds  since  neurons 
operate  with  ionic  conduction.  Artificial  semiconductor  based  neurons 
operate  with  electronic  conduction  and  hence  can  be  made  considerably  faster 
with  time  constants  approaching  nano-seconds.  Collective  processing  such 
artificial  nets  can  therefore  be  extremely  fast.  The  remarkable  ability  of 
neural  nets  in  handling  sketchy  (erroneous  or  incomplete)  information  and 
their  fault  tolerance  (graceful  degradation  in  performance  with  element 
failure)  make  them  particularly  attractive  in  pattern  recognition,  robotics, 
and  autonomous  system  intended  to  operate  virtually  unattended  for  long  time 
periods . 

In  this  report  we  describe  progress  during  the  past  year  in  our  study 
of  optical  computing  based  on  the  Hopfield  model  of  neural  networks  and 
other  similar  models.  Optical  implementations  are  of  interest  because  they 
furnish  the  parallelism  and  massive  i  ri  t  or  co  n  nec  t  i  v  i  t  y  required  in 
implementing  neural  net  models.  At  the  same  time  neural  net  models  provide 


1 .  INTRODUCTION 


Interest  in  neural  network  models  (see  for  example,  [ 1  ] - [ 9 ] )  end  their 
opto-electronic  analogs  stems  from  well  recognized  capabilities  of  the  brain 
and  the  fit  between  what  optics  can  do  and  what  even  simplified  models  of 
neural  nets  can  offer  toward  the  development  of  new  approaches  to  collective 
signal  processing.  We  know  that  the  brain  is  not  as  good  in  arithmetic 
operations  as  a  digital  computer  but  when  it  comes  to  operations  such  as 
association,  categorization,  generalization,  c 1  as s  i  f  i  c  t  i  on ,  feature 
extraction,  recognition,  and  optimization  it  outperforms  even  the  most 
powerful  of  todays  computers .  The  brain's  amazing  capabilities  in  analyzing 
sensory  data  and  in  controlling  motor  function,  let  alone  complex  thought 
and  intelligent  reasoning,  makes  it  an  intriguing  model  for  smart  sensing 
and  automated  recognition  systems  and  for  robotic  and  automatic  control 
systems  with  unescapable  ramification  for  pattern  recognition,  artificial 
intelligence  and  autonomeous  systems.  An  interesting  aspect  of  its  ability 
in  processing  sensory  data  is  the  ease  with  which  it  solves  computationally 
complex  problems  associated  for  example  with  vision  that  are  basically 
inverse  problems  [10].  These  are  computationally  vexing  because  of  their 
i  1 1- posedness  [11].  The  brain's  associative  memory  capabilities  where 
nearest  neighbor  searches  are  performed  successfully  even  when  the 
information  it  is  presented  with  is  sketchy  are  evidence  of  its  remarkable 
robustness  where  high  levels  of  missing  or  erroneous  data  in  the  input  can 
be  tolerated.  The  ability  of  the  brain  to  supplement  missing  information 
has  exciting  implications  for  super-resolution  and  other  similar  problems  of 
signal  recovery  from  incomplete  and  noisy  data  [11], [12].  The  capabilities 
of  the  brain  in  rapid  solution  of  optimization  problems  [13]  are  also  well 


appreciated.  Add  to  the  above  that  the  brain  is  amazingly  fault  tolet ant  as 


its  (cells  or  neurons),  unlike  cells  in  other  parts  of  the  body,  are  not 
renewable.  Yet,  despite  loss  of  a  ncn-negligible  number  of  cells  by  the 
time  one  of  us  reaches  the  age  of  50  we  continue  to  function  normally.  We 
know  something  about  how  the  brain  processes  information.  We  know  that  it 
does  that  in  parallel  or  more  precisely  collectively  by  means  of  massively 

interconnected  networks  of  neurons.  There  are  anywhere  between  1010~  1011 

3  4 

neurons  in  the  brain,  each  making  about  10  -  10  synaptic  interconnections 

13  15 

with  neighboring  neurons  for  a  total  of  10  to  10  interconnections.  Even 
when  we  assume  the  neuron  to  take  only  two  states:  firing  or  not  firing 
i.e.,  binary  neuron,  the  total  number  of  degrees  of  freedom  of  the  the  brain 

io15 

is  truly  astronomical  reaching  2  .  Each  neuron  independently  evaluates 
its  state  and  decides  to  change  it  or  not  depending  on  whether  the  sum  of 
its  synaptic  (exitatory  and  inhibitory)  inputs  exceeds  a  given  threshold  or 
not  performing  thus  a  highly  nonlinear  (logic)  operation. 

Massive  connectivity  and  parallelism  are  the  two  main  attributes  of 
optics.  Optics  can  therefore  play  an  important  role  in  the  implementation 
of  models  of  neural  nets  for  computing  and  signal  processing.  Besides 
developments  in  programmable  nonvolatile  spatial  light  modulators  [1U], 
O’  ;icai  light  amplifiers  [15],  and  optical  bistability  devices  [16]  promise 
to  play  a  useful  role  in  the  implementations  of  programmable  connectivity 
matrices,  and  optical  decision  making  elements  leading  ultimately  to 
powerful  neural  net  type  processors. 

The  above  observations  have  served  as  motivation  for  several  workers 
I  1  7 ] - [ 3 1  ]  bo  investigate  the  feasibility  and  capabilities  of  optical 
implementations  of  neural  nets.  Optical  implementations  of  1 -D  and  2-D 
distributions  of  neurons  (neural  nets)  have  been  considered  and/or  studied 
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and  evaluated  in  coherent  and  incoherent  light.  The  performance  of  such 


networks  is  found  to  conform  to  theoretical  prediction  of  storage  capacity 
(number  of  entities  that  can  be  reliably  stored)  [32]— [3^3 ,  and  exhibits 
robustness  and  fault  tolerance.  In  an  optical  implementation  of  an 
associative  memory  of  N=32  neurons,  with  three  stored  entities  errors  of  up 
to  30?  in  the  input  stimulus  was  tolerated  during  complete  and  correct 
recall  of  the  nearest  neighbor  stored  entity.  Furthermore,  accidental 
failure  of  about  10?  of  the  neurons  hardly  caused  noticable  degradation  in 
performance.  Some  of  the  above  studies  have  been  aimed  at  finding  means  of 
simplifying  optical  implementation  of  large  neural  nets,  increasing  the 
storage  capacity,  and  finding  ways  for  their  interconnections  as  modules 
into  more  complex  systems  to  perform  higher  order  hierarchical  processing. 

It  is  worth  noting  that  operations  such  as  associative  recall  and 
nearest  neighbor  search  performed  in  the  above  systems  can  be  carried  out  by 
more  conventional  means  without  resort  to  neural  net  processing.  What  does 
the  neural  net  approach  then  offer  that  conventional  signal  processing  and 
comDuting  methods  do  not?.  The  answer  to  this  question  may  be  found  in  the 
following  observations:  (a)  Neural  nets  and  their  models  provide  us  with  a 
new  way  of  viewing  signal  processing  and  computing  problems  that  may  lead  to 
solutions  and  applications  not  thought  of  otherwise  in  a  manner  very  much 
reminiscent  to  what  happened  with  the  advent  of  holography  several  decades 
ago.  !b)  The  brain  and  its  neural  organization  are  the  results  of  a 
prolonged  evolutionary  process  in  which  only  those  permutations  that 
enhanced  the  survivability,  and  by  implication  the  function,  of  the  organism 
have  been  retained  and  all  other  permutations  have  been  discarded  through 
''survival  of  the  fittest"  process.  The  result  is  a  biological  "computer" 


that  has,  as  pointed  out  earlier,  no  equal  at  present  among  artificial 


systems  when  it  comes  to  tasks  of  recognition,  classification, 
categorization,  generalization,  recognition,  and  optimization.  It  is  not 
surprising  therefore  to  see  the  current  interest  in  brain  mechanisms  in  the 
artificial  intelligence,  cognitive  science,  and  pattern  recognition 
communities.  Even  early  researchers  in  computer  science  were  concerned  with 
brain-related  attributes  of  computing  systems  [35].  A  serial  approach  to 
computing  was  however  adopted  and  pursued  because  of  better  understanding, 
easier  mathematical  modeling,  and  possibly  because  a  collective  or  parallel 
approach  to  computation  would  have  been  technologically  and  economically  not 
feasible  at  the  time.  There  is  much  therefore  that  we  can  afford  to  learn 
from  the  brain  and  its  neural  nets  that  can  prove  to  be  useful  in  artificial 
man  made  systems.  The  general  idea  here  is  not  to  attempt  to  build  systems 
that  fully  emulate  the  brain  in  all  its  functions,  as  this  would  be 
unrealistic,  but  to  gain  insights  in  its  operation  and  mechanisms,  as  to 
allow  us  to  be  able  to  glean  information  about  these  attributes  and 
functions  that  would  be  worth  incorporating  in  man  made  systems.  Optical 
analogs  of  neural  nets  can  be  a  useful  tool  in  gaining  such  insights  and  as 
a  way  of  ultimately  simulting  neural  nets  consisting  of  thousands  to 
millions  of  elements,  (c)  Information  processing  in  the  brain  can  be 
characterized  as  collective,  adaptive,  highly  nonlinear,  and  relying  heavily 
on  feedback.  We  know  that  all  these  are  attributes  of  powerful  signal 
processing  algorithms.  What  is  amazing  is  that  these  capabilities  are 
achieved  in  networks  that  appear  to  be  homogeneous  in  their  general 
structure  i.e.,  only  a  few  different  types  of  neurons  (cells)  are  involved 
and  the  process  followed  by  most  neurons  is  macroscopieally  similar  in  the 
sense  that  each  neuron  receives  inputs  from  other  neighboring  neuron  and 
decides  to  change  its  state  or  not  depending  on  the  nature  of  the  inputs  and 
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the  synaptic  weights  they  make  with  the  neuron  where  memory  is  stored.  In 


other  words  the  brain  exhibits  a  fractal  (self-similar)  feature  in  its 
organization.  It  is  now  recognized  that  the  same  basic  structure  of  highly 
interconnected  neurons  can  be  involved  in  the  formation  of  signal-adaptive 
self-organized  networks  for  unsupervised  learning  and  feature  extraction 
from  sensory  data  [H],[36],  in  the  formation  of  associative  memory  modules 
tuned  to  respond  to  specific  features  of  the  sensory  data,  C37]  —  r-4i  h  and  for 
recognition  and  perception  "circuits"  of  the  brain.  This  is  a  tremendous 
flexibility  that  is  achieved  with  little  apparent  specific  design  unlike 
conventional  circuit  design  where  function  specific  components  and  parts  are 
connected  in  accordance  to  specific  rules  to  perform  required  signal 
processing  tasks.  As  evidence  of  this  flexibility  it  is  worth  noting  in  this 
regard  that  in  associative  memory  based  on  neural  net  models  the  function  of 
memory  and  signal  processing  involved  in  recall  are  totally  intermingled, 
(d)  The  fractal  nature  of  neural  nets,  their  robustnewss  and  fault  tolerance 
can  be  immensely  useful  for  modern  VLSI  technology.  Continuing  advances  in 
microf abrication,  and  optical  technologies  promise  to  make  it  possible  to 
fabricate  large  numbers  of  massively  interconnected  decision  making 
switching  elements  with  low  power  consumption.  Neural  net  models  and 
ar  :h  i  he ures  vin  provide  such  structures  (possibly  in  the  form  of  opto- 
e  1  -•of rori  1  0  chips'  with  the  robustness  anc  the  fault-tolerance  badly  needed 
:  n  VLJ I  to  ill*".-;  >te  the  central  problem  of  yield.  Imperfect  chips  may  no 
longer  be  regarded  is  rejects  but  can  find  use  in  artificial  neural  net 
pro  '-nsors  or  'ompufers .  e  Arohitectures  and  Optical  implementations  of  2- 


a. 


rotation,  and  size  invariant  classifiers  or  feature  spaces)  derived  from 


sensory  array  data  generated  for  example  by  electromagnetic  or  acoustic 
(including  seismic  and  sonar)  receiver  arrays.  Associative  memory  modules 
can  tell  us  then  rapidly,  in  a  matter  of  a  few  time  constants  of  the  neuron- 
like  elements  used,  which  member  of  an  ensemble  of  input  descriptors  is 
recognized  by  the  memory  even  when  the  descriptor  information  it  is 
presented  with  happens  to  be  incomplete  or  sketchy.  Recognition  of  an  input 
is  manifested  by  the  neural  net  converging  and  settling  into  one  of  its 
nominal  states  corresponding  to  that  stored  entity  which  most  resembles  the 
input.  It  is  clear  that  this  operation  by  itself  may  not  be  useful  unless 
some  means  for  identifying  or  recognizing  the  outcome  of  the  nearest 
neighbor  search  is  incorporated.  This  suggests,  in  the  absence  of  such 
"recognizer",  that  auto-associative  memories  may  be  useful  as  building 
blocks  of  more  complex  processors  in  which  higher  order  hierarchical 
processing  takes  place,  in  a  manner  not  yet  fully  understood.  Such 
processing  would  have  the  aim  of  first  reducing  the  information  content  of 
the  signals  flowing  concurrently  through  such  a  network  of  associative  CAMs 
and  then  fusing  their  outcomes  into  a  single  meaningful  concept  or 
perception.  In  biological  systems,  such  perception  is  thought  to  be 
associated  witn  a  prescribed  trace  of  neural  activity  or  spatio-temporal 
firing  pattern  in  the  cortex.  Associative  memory  modules  by  themselves 


gful  when  used  in  a  heter c-assce i at i 
scrip-tors  or  classifiers  is  not  stored 
■:iat  ion  with  reference  entities  that 
Terence  entities  that  are  compatible  wi 
s  or  pictorial  representations  ir 
i  analog  or  digital  outputs  suitabl 


ve  mode  where  the 
by  auto-association 
.  are  more  e  a  s y  t o 
th  the  "recognizer" 
case  of  a  human 
act i voting  meter 


I 


-10- 


o.  ft  C 


functions  in  robotic  and  artificial  intelligence  systems.  Hetero- 
associative  storage  and  recall  of  partial  information  can  play  an  important 
role  in  smart  sensing  and  automated  recognition  systems  as  was  recently 
demonstrated  with  an  example  of  identifying  different  types  of  aircraft  from 
partial  information  about  their  sinogram  classifiers  [M2]  and  individual 
faces  from  edge  enhanced  versions  as  classifiers  [ U 3 ] . 

2.  RESEARCH  ACCOMPLISHMENTS 

Neural  net  models  and  their  optical  analogs  provide  a  new  approach  to 
signal  processing.  The  aim  of  our  research  activity  during  the  period  of 
this  report  was  therefore  to  study  and  assess  the  capability  of  neural  net 
processing  in  actual  challenging  applications  where  they  are  expected  to  be 
most  useful  such  as  automated  recognition.  To  this  aim  we  have  also  paid 
attention  to  the  question  of  optical  implementation  of  large  neural  nets  of 
the  type  we  expect  is  needed  in  smart  sensing  and  automated  recognition 
applications.  Therefore  on.  research  efforts  during  the  preceeding  period 
have  focused  specifically  upon  the  following  tasks: 

(a)  Study  of  methods  for  simplifying  optical  implementations  of  neural 
net  models  without  sacrificing  performance.  To  this  purpose  a 
systematic  numerical  study  of  optical  embodiments  that  employ 
different  types  of  synaptic  or  connectivity  masks  (e.g., 
multivalued,  ternary,  unipolar  binary)  and  a  variety  of 
thresholding  methods  (e.g.,  zero  thresholding,  adaptive 
thresholding  where  the  threshold  is  proportional  to  the  energy  of 
the  initiating  and  iterated  input,  adaptive  thresholding  and 
relaxation  where  the  neuron  is  influenced  by  its  previous  state  - 
self  inhibition)  were  studied  and  their  performance  compared.  The 


results  of  this  study  are  detailed  in  Appendix  I  of  this  report. 
They  show  that  the  adaptive  thresholding  and  relaxation  scheme  can 
be  used  with  a  unipolar  binary  (u.b.)  mask  without  suffering  any 
degradation  in  performance  as  compared  to  the  ideal  case  when  zero 
thresholding  and  a  multivalued  mask  are  employed.  The  ability  to 
use  a  u.b.  connectivity  mask  enables  their  realization  in  black- 
and-white  optical  transparencies  and  opens  the  way  for  the  use  of 
nonvolatile  spatial  light  modulators  for  computer  driven 
programmable  connectivity  matrices. 

(b)  Numerical  simulation  and  experimental  study  of  optical  analogs  of 
2-D  distributions  of  neural  nets  (2-D  neural  nets)  based  on  a 
novel  scheme  for  partitioning  the  resulting  A-D  connectivity 
matrix  into  a  2-D  array  of  2-D  matrices  was  undertaken. 
A r ch  i  t  e  c  t ur es  based  on  this  matrix  partitioning  scheme  were 
studied  numerically  and  found  to  work  efficiently.  The  primary 
motive  for  our  interest  in  2-D  neural  nets  is  their  compatabi 1 i ty 
with  2-D  feature  space  data  and  their  potential  for  realizing 

dense  neural  nets  of  -(10  -10  )  elements.  The  results  of  this 
task  are  detailed  in  Appendix  II  of  this  report.  There  the 
application  of  a  2-D  neural  net  of  32x32  elements  in  an  automated 
recognition  situation  namely,  recognition  of  scale  models  of  three 
aero-space  targets  employing  sinogram  representations  or  feature 
spaces  of  these  targets  produced  employing  a  unique  microwave 
measurement  facility  is  described  in  some  detail.  Supei — resolved 
recognition  employing  partial  information  (r>20%)  of  the  sinogram 
data  of  the  three  test  objects  and  making  use  of  a  hetero- 
associative  memory  whose  output  at  the  completion  of  a  successful 


search  is  a  label  of  the  corresponding  recognized  object  is 

demonstrated.  The  use  of  realistic  data  in  this  work  is  important 

as  it  illustrates,  together  with  the  ability  to  recognize  the 

input  when  it  is  partial  and  sketchy,  the  unusual  robustness  of 

neural  net  models  and  their  analogs.  The  results  presented  in 

Appendix  II  exhibit  also  a  certain  degree  of  generalization  when 

the  memory  output  is  a  composite  of  the  entities  it  was  taught. 

This  occurs  only  if  the  amount  of  information  it  is  presented  with 

is  meager  (e.g.,  10?  or  less  of  a  full  sinogram  pattern).  These 

results  seem  to  underline  the  uniqueness  of  the  neural  net 

approach  to  signal  processing  in  that  complex  functions  are 

realized  in  self  similar  nets  of  interconnected  neurons  as  opposed 

to  conventional  signal  processing  schemes  and  architectures  based 

on  a  highly  organized  modular  approach, 

(c)  Initiation  of  construction  of  a  2-D  neural  net  of  5x5  element  to 

be  used  as  a  test  bed  leading  to  eventual  construction  of  1 6 x  1 6 

and  then  32x32  neural  nets.  All  of  these  embodiment  utilize  a 

lenslet  array  to  multiplex  the  initiating  external  input  and  the 

iterated  input  onto  the  partitions  of  the  A-D  connectivity  matrix 

T.  .  and  nonlinear  electronic  feedback  loops  from  an  output 
ljkl 

monitoring  photodetector  array  paired  with  an  LED  array  at  the 
i nput . 

Collective  processing  is  recognized  to  be  an  effective  tool  in  the 
solution  of  combinatorial  problems.  Both  neural  net  based 
algorithms  and  similar  simulated  annealing  algorithms  are 
applicable.  We  have  initiated  a  study  of  this  important  area  by 


(d) 


considering  the  novel  problem  of  synthesizing  an  optimal  radiation 


pattern  cf  a  phased  array  antenna  by  simulated  annealing.  The 


results  of  this  study  are  detailed  in  Appendix  III.  The  results 
snow  that  simulated  annealing  can  be  a  useful  tool  in  phased  array 
synthesis.  The  attractiveness  and  effctiveness  of  the  method  can 
be  considerably  enhanced  if  the  computation  times  required  are 
reduced.  Hybri  (  opto- d i gi tal  or  opto-electronic)  schemes  can 
play  a  role  here.  Their  applicability  is  being  investigated. 

3.  CONCLUDING  REMARKS 

At  this  stage  of  their  early  development  one  can  only  venture  to  say 
that  neural  net  models  offer  a  new  and  intellectually  stimulating  approach 
to  computing  and  information  processing.  The  approach  "dove-tails"  very  well 
with  the  capabilities  of  optics  and  compliments  them.  While  optics 
naturally  provides  the  parallelism  and  massive  interconnectivity  needed  in 
the  implementation  of  neural  nets  and  their  models,  these  on  their  part 
provide  the  robustness,  fault  tolerance,  and  the  power  of  nonlinear 
processing  and  feedback  that  are  generally  lacking  in  optical  processing. 
The  combination  results  in  systems  that  can  no  longer  be  characterized  by 
the  usual  measures  of  convolution  and  impulse  response  (variant  or 
invariant)  because  superposition  no  longer  applies  but  may  have  to  be 
character  i  zed  instead  by  stable  states  or  modes  in  the  N-dimensiorial  phase- 
space  of  an  N-neuron  network.  Optical  analogs  of  even  highly  simplified 
models  of  neural  nets  exhibit  a  high  degree  of  robustness  and  fault- 
tolerance,  and  can  be  implemented  as  content  addressable  associative 
memories  for  use  in  computers  and  in  smart  sensing  and  automated  recognition 
systems,  or  as  networks  for  rapid  collective  solution  of  computationally 
complex  tasks  encountered  in  optimization,  vision,  and  inverse  problems. 
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They  may  also  provide  a  tool  for  the  study  of  nonlinear  dynamics,  chaos,  and 
even  prove  to  be  useful  in  clinical  studies  of  mental  disorder. 
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CONTENT  ADDRESSABLE  MEMO R  f  HITH  SMOOTH  TRANSIT  I  ON 
AND  ADAPT  I'-'E  THRESHOLD  I NG 

Kanq  S.  Lee  and  Natal  H.  Far  hat 
U  n  1  '  e  r  s  1  t  v  o  f  P  e  n  n  s  y  1  '  a  n  1  a 
The  Electro-Optics  and  Microwave-Optics  Laboratory 

£  0 0  S .  3 3 r  d  street 
Philadelphia,  Pa  19104 

ABSTRACT  :  The  use  of  unipolar  binary  masks  and  adapt io 
smooth  thresholding  in  optical  implementations  of  neura 
nets  appears  to  combine  the  better  performance 


of  multi-leue 


transition  method  is  also  related  to  the  relaxation  techniques 
used  in  solving  systems  of  linear  equations.  Result  a  of 
digital  simulation  are  presented  for  a  32  neuron  S"  stern. 

These  are  shown,  as  far  as  storage  capacity  and  error  correction 
are  concerned.  to  compare  favorably  with  performance  of  a 
multi-  v  alue  d  rn  a  s  k  w  here  merel  v  sharp  thres  h  o  1  d  1  n  g  is  used. 
Freliminar  1  experimental  results  obtained  with  a  modified 
version  of  a  previously  reported  32  neuron  optical  netuorn. 
are  also  given. 


2.  OPTICmL  IMPLEMENTS  I  ON  OF  CONTENT  ADDRESSABLE 
MEM OP f  f CAM  1  USING  A  BIPOLAR  BINaRi  MASK 


E  e' 1  e  r  a  1  s  cherries  for  optical  1  rnp  1  ernen  t  a  1  0  n  of  a  Ci~M  based 
on  the  Hopfield  model  [1]  and  other  similar  models 
-a"e  been  described  earlier  [4 ]-[?].  In  one  of  the  implement a- 
i:ns  an  arrau  of  light  emitting  diodes  1  LEDs  1  is  used  to 
,'epvesent  the  logic  elements  or  neurons  of  rhe  network  . 

T-eir  state  <  on  or  off  1  can  represent  unipolar  binary  actor; 
f'a*  are  stored  in  the  memory  matri  ,  employing  the  usual 

s -or  age  recipe  [1].  Global  inter-connection  of  the  elements 

1 s  realized  as  shown  in  Fig.  1'  a1  through  re  addition  of 

nonlinear  feedback  1  thresholding,  gain,  and  feedback  '  to 
a  :  r  "  en  1 1  0  r.  ai  optical  "ec  1 0  r  -rr.a  t  r  1  1  multiplier  [3]  in  which 

*  e  a  rrj..  0  r  LEDs  represents  the  input  vector  and  an  arra1 

:  *'  photodiodes  -  PC' 1  is  used  to  detect  the  output  vector. 

The  PC’  array  output  is  thresholded  and  fed  back  in  parallel 

*  :  drwe  the  corresponding  elements  of  the  LED  arra".  Multi¬ 

plication  of  the  input  ''ector  bv 
b-  horizontal  imaging  and  "ertical  smearing  of  the  input 

"ector  that  is  displayed  bv  the  LEDs.  on  the  plane  of  the 


t he  T! n  matri  -  1  s  achie-ed 


m  ask.  his  is  1  rnp  i  ernen  ted  b1 1  me  an  s 

S  "  S  t  erri 


an  an  amorphic 
A  sec  0  n  d 


1  0 

1  en  a 


o m 1 1 1  ed 


f  r  o  m  F  1  g 


1 '  a  1  f  0  r 


s  1  rnp  licit 


an  amor  phi  c  lens  system  ■:  also  not  shown  t  is  used  to  collect 
t  he  1  1  gh  t  erner  g  1  n  g  f  r  om  e  ach  r  ow  o  f  the  T  ^  rn33  k  0  n  j  nd  t ,,  l  du  3l 
photo -sites  of  the  F‘D  array.  In  this  scheme  a  bipolar  T 

1  i 

matri  is  realized  in  incoherent  light  bv  dividing  each  row 
of  the  T  matrix  in  two  subrows,  one  for  positive  and  one 
for  negative  values  and  bringing  the  light  erner  ginq  from 
each  subrow  to  focus  on  two  adjacent  photo -sites  of  the  pO 
array  that  are  electrically  connected  in  opposition  as  depicted 
in  Pig.  1.  In  the  ev stem  shown  in  Fig.  1,  feedback  is  achieved 
b 11  electronic  wiring.  It  is  possible  and  preferable  t  o  d  i  s  p  o  s  e 
of  electronic  wiring  altogether  and  replace  it  with  optical 
feedback  .  This  can  be  achieved  bv  combining  the  PD  and  lED 
arrays  in  a  single  hybrid  or  monolithic  structure  that  can 
also  be  made  to  contain  all  ICs  for  thresholding,  amplification, 
and  driving  of  LEDs.  Optical  feedback  becomes  even  more 

attractive  when  we  consider  that  arravs  of  nonlinear  optical 
light  amplifiers  with  internal  feedback  [?]  or  optical  bi¬ 
stability  devices  ( OBDs  :•  [10]  can  be  used  to  replace  the 

PD  LED  arra".  This  can  lead  to  simple  compact  Crtfi  structure 
that  rnav  be  interconnected  to  perform  more  sophisticated 

computations  than  the  nearest  neighbor  search  performed  bv 
a  single  ChM . 

The  detail  and  operation  of  a  simple  optical  S"Stern 
simulating  a  network  of  M=32  neurons  that  is  a  variation 
of  the  scheme  presented  in  Fig.  1  has  been  reported  in  [7]. 
Here  we  reMiew  briefly  details  of  this  system  that  are  relevant 
t  o  the  subject  matter  of  this  paper. 

The  systems,  details  of  which  are  given  in  Fi qs .  3-8, 

was  constructed  with  an  array  of  32  LEDs  and  two  multichannel 
silicon  PC'  arrays,  each  consistinq  of  32  elements.  Twice 
as  man"  PC'  elements  than  LEDs  are  needed  in  order  to  implement 
a  bipolar  memory  mask  transmittance  in  incoherent  light  in 
accordance  to  the  scheme  of  Fig.  lib'.  In  this  scheme  a 

bipolar  b  l  n  a  r "  '  E  b  T  ^  mask  <  < a  s  utilized.  The  mask  was 

prepared  for  f  1  =  3  b  l  n  a  r  ■ 1  s  t  a  t  <=■  1  •  ®  c  t  o  r  =  .  T  h  e  three  ■ .'  e  r  t  n  r  = 


that  rnav  be  in 


or  words  chosen.  their  Hammim  distances  from  each  other. 


and  the  result inq  T 


memo  r  v 


matrix  are  reproduced  in  Fiq 


The  mean  Hamminq  distance  between  the  three  vectors  is 


l>o.  A  binary  photographic  transparency 


32xfe4  square  pixels 


was  compu  ter  gener  a  ted  f  rom  the  T  ^  ^  rnatr  1  x  bo  as- si  gn  1  nq  the 
P o s 1 1 1 y e  values  in  any  g l y e n  r o w  of  T  to  transparent  pixels 
in  one  subrow  of  the  mask  and  the  negative  values  to  transparent 
pixels  in  the  adjacent  subrow.  To  insure  that  the  image 
of  the  input  LED  array  is  uniformly  smeared  over  the  memory 
m a s k  i t  w a s  f o u n d  c o n y enient  to  split  the  m a s kin  t w o  h a 1 y e s . 


shown  in  Fiq. 


and  to  use  the  resulting  submasks  in 


two  identical  optical  arms  as  shown  in  Fig.  4.  The  size 
of  the  subrows  of  the  memory  submasks  was  made  exactly  equal 
to  the  element  size  of  the  PD  arrays  in  the  vertical  direction 
which  were  placed  in  register  against  the  masks.  Light  emerging 


from  each  vertically  oriented  subrow 


a  memory 


i-ubmask 


was  collected  (spatially  integrated.'  bv  one  of  the  vertically 


oriented  elements  of  the  multichannel  PD  arras 


In  this 


fashion  the  an  amorphic  optics  required  in  the  output  part 
of  Fig.  1(a)  are  disposed  of,  resulting  in  a  more  simple 
and  compact  system.  Pictorial  views  of  the  input  LED  array 

and  the  two  submask  PD  array  assemblies  are  shown  in  (a' 


snd  (b)  of  Fiq. 


respectively.  All  electronic 


.  l  r  c  u  l  t  < 


(amplifiers.  thresholding  comparators,  LED  drivers  etc.) 
in  the  32  parallel  feedback  channels  are  contained  in  the 
electronic  amp lifi cation  an d  t hr esho 1 d 1 n g  bo x  shown  in  Fig. 


4 1  a)  and  in  the  boxes  on  which  the  LED 


and  the  t '  j  o 


submask  PD  array  assemblies  are  mounted  'see  Fiq. 


A  pictorial  v i ew  o  r 


in  Fiq.  6 .  This 


contains  an  a r r  a n q em e n t  or 


s  w itches  an  d 


a  32  element  LEE'  display  panel  whose  elements  are  connected 


in  parallel  to  the  input  LED  array. 


'he  f  u net l o n  o  f  this 


box  is  to  compose  and  display  the  binary  input  word  or  vector 


that  appears  on  the  input  LED  arra"  of  the 
Fiq.  ji  a  i  . 


=tem  shown  in 


u  ector 


selected 


i  J  nee  an  input 


i t  ap pear s  d 1 s p 1  ay ed 


on  the  composing  box  and  on  the  input  LED  box  simultaneously. 

m  single  switch  is  then  thrown  to  rc]*ase  the  system  into 

o  p  e  r  a  t i o  n  w  l  t  h  the  c o  m p  o  s  e  d  y ector  as  the  initializing  v ector . 
The  final  state  of  the  system,  the  output,  appears  after 

a  f  ew  l terati o  n s  d i sp 1  ay  ed  on  the  i  n p  u  t  LED  ar r  ay  an  d  d i s  p 1 av 
box  simultaneously.  The  above  procedure  provides  for  convenient 
exercising  of  the  system  in  order  to  study  its  response  vs, 
stimulus  behavior.  An  input  vector  is  composed  and  its  Hamming 
distance  from  each  of  the  nominal  state  vectors  stored  in 
t  he  memo r  y  is  not  ed .  The  v ec  tor  is  t  hen  u  sea  to  initialize 
the  CAM  as  described  above.  The  output  vector,  representing 
the  final  state  of  the  CAM  which  appears  almost  immediately 

on  the  display  box,  is  e  arnined.  The  response  time  of  the 
electronic  feedback  channels,  as  determined  bv  the  3  dB  roll-off 


o  f  t  he  amo  1  i  f  i  er  s  ,  was  abo  u  t  60  rn i  1 1  i  seco n ds  , 


Speed  of  the 


operation  was  not  an  issue  in  this  study,  and  thus  the  slow 
response  time  was  chosen  to  facilitate  the  experiment. 

The  above  implementation  of  CAM  based  on  the  Hopfield 
model  for  neural  nets  indicate  the  op to-electr on l c  analog 


o  f  two  neu  r  o  n  s  l  n  sy  n  ap  t  i  c  co  n  t  ac  t  shown  in  na. 


In  r  h  i  s 


scheme  the  presence  of  both  excitatory  and  inhibitor"  sMnapses 
reouire  the  use  of  twice  the  number  of  neurons  implemented 
i  n  p  h  o  todetset  o  r  s  and  r  o 1  ->  s  of  the  optical  m  ask  in  :ra*r  t  o 


realize  the  bipolar  interconnection  matri 
1 1  q  n  t  . 


in  it:  c  o  h  e  -  e  n 


The  results  of  e  ercising  and  evaluating  trie  oe-t'orma: 
o  f  t  hi  e  Cam  described  above  w  ere  presented  in  [  ~  ]  .  T  h  < 
show  that  the  system  is  capable  of  associative  recall.  " 
error  correction  capabilities  and  is  extremely  robust, 
is  obvious  that  considerable  simplification  can  be  achie1 


in  the  a b o v e  l rn p  1  ern entati  o n  i  f 


unipolar  binar"  mask  is 


perm i tted 


An  i mm e d l a  t e 1 v  o b " l o u  s  a d v an  t  aqe  of  this  i  s  that 


the  same  number  of  pho to detectors  used  in  the  preceding  system 


can  be  used  to  i m p lement 


C AM  with  64  neurons  instead 


w i t h  some  add  1 tio n al  comp o n en  t s  an d  a  new  u nipolar  binar v 
(  UB )  mask.  For  this  reason  we  have  carried  out  a  stud1,1  of 
me t  ho  ds  f  o r  1  mp I ernen  tinq  CAMs  ernp  1  o v  1  n q  u n  1  p o lar  binar y  T 

i 

matrix  wh i ch  ar e  descr i bed  in  the  foil ow i n g  sec t ions . 

3.  CONTENT  ADDRESSABLE  MEMOR  ,  WITH  UNIPOLAR. 

B  I N  A  P.  r  I  NT  ERG  ONN  E  CT  I  ON  MAT  P  I  X 

The  use  of  UB  interconnection  matrix  or  synaptic  mask 
ns;  the  a  d " a  n t  a  g  e  ,  pointed  out  earlier,  of  further  simplifying 
optical  implementation  of  neural  networks  and  of  opening 

the  way  for  the  stud”  of  larger  networks.  It  also  has  the 

advantage  of  facilitating  the  use  of  recyclable  spatial  light 
modulators  i  SLMsw  such  as  the  Litton  non-volatile  magneto -op t l c 
S LM  i M 0 - S  LM  i  [11]  w hi ch  is  easiest  to  u s  e  a s  a  p r  o  g  r  a mm  able 
mask  i  t  ^  matrix  i  in  a  UB  transmission  mode.  The  use  of 
a  modifiable,  external  memory  driven,  MO-SLM  as  a  T  mask 
enables  the  same  Cam  to  search  throuqh  a  large  file  of  T 

i  i 

matrices  residing  in  an  external  store  in  a  relatively  short 

time.  With  such  masks  a  non -zero  threshold  level  must  be 

adopted  as  the  inhibitory  signals  to  a  neuron  i or  to  the 

pair  of  p  h  o  to  - diodes  in  Fig.  7  t  b  >  )  arising  f  r  o  rn  negative 

"alues  of  the  T  mask  are  now  absent.  Two  schemes  of  adaptive 

t  hr  esho  ldm  g  wer  e  ex  am  l  n  ed  .  On  e  is  shown  in  Fig.  3  .  In  this 

scheme,  suggested  bv  the  observation  that  dormant  neurons 

have  a  non-zero  firing  rate,  we  adopt  one  half  the  energy 

E  =  1/2  I"  of  the  UB  input  vector  as  a  non-zero  adaptive 
J  -1 

thresholding  level  for  the  comparator  C.  The  comparator 

output  which  can  be  positive,  zero  or  negative  depending 
on  whether  the  weighted  projection  v  =  T  t  . .  of  the 

l  j  li  i  - 

input  vector  is  greater.  equal  or  less  than  E  is  applied 
to  a  zero  level  thresholding  element  the  output  of  which 
is  use  •  to  dr i "e  the  corresponding  element  in  the  input  LED 


V. 


vyv 


The  second  scheme  examined  adds  relaxation  to  the  preceding 
scheme  as  shown  in  Fig.  .  As  each  iteration  proceeds  in 


t  h  i  s 

s  cheme , 

the 

c  o  m  par 

a  t  o  r  output,  the 

differ  en  ce 

h i=*  t  t.j ppn 

the 

w  sighted 

p  r  o  .1 

ec  1 1  o  n 

of  the  current 

iterate  and 

one  half 

of 

the  energ 

>  o  f 

t  he 

input  vector,  i  s 

at tenuated 

t  h  r  o'jqh 

rn  u  1 1 

l  p  1 1  c  a  1 1  o  n 

b'v» 

a  P  o  s 

l tive  relax  at i on 

P  ar  ame t  er  a 

o  f  v  a  1  u  e 

less  than  uni  tv.  The  outcome  is  added  to  the  value  of  the 

previous  iterate,  retained  through  the  secondary  feedback 
loop  with  time  d e 1  a v  f  and  the  result  is  applied  to  a  limiter. 
Since  the  initializing  input  vector  has  a  value  of  one  or 
zero,  the  negative  value  and  value  of  greater  than  one  are 

clipped  while  the  values  between  zero  and  one  are  remained 
unchanged  through  the  limiter.  The  limiter  output  is  used 
to  drive  the  cor  esponding  LED  in  the  input  LED  arrav.  In 
this  fashion  a  smooth  transition  of  states  in  the  CAM  is 
realized.  The  technique  bears  resemblance  to  the  relaxation 
method  used  in  solution  of  systems  of  linear  equations. 

The  results  of  numerical  simulation  of  the  above  two 
schemes,  assuming  a  unipolar  binarv  version  of  the  T .  ^  matrix 
and  three  stored  vectors  are  shown  in  the  last  two  columns 
of  table  I.  These  are  compared  with  the  results  of  simulations 
for:  1 i  a  grey  level  mask  with  zero  thresholding,  2)  bipolar 
b l  n ar  y  '.-1,0,1  1  mask  w l  t h  zero  t hr esho  1  d l  n g  ,  an d  3  i  u n  i  p o  1  ar 
binarv  (0.1)  mask  with  the  mean  of  the  projections, 

p  r  o  J  >  =  1 .  M  Z  v  s e  1  e c  ted  as  thres h o  1  d  l  ti  g  y  a  1  u  e  .  The  results 

j  J 

are  shown  in  the  other  columns  of  table  I.  The  numbers  in 
the  table  represent  the  maximum  Hamming  distance  of  an  input 
vector  chosen  to  exercise  the  memory  from  the  stored  "ector 
that  can  be  corrected.  The  exercising  input  vector  is  obtained 
from  a  stored  vector  by  switching  digits  from  its  end.  The 
symbol  x  in  the  table  means  that  particular  input  vector 
is  not  recognized  even  when  none  of  its  digits  differ  from 
the  from  the  stored  v ector.  The  si m u 1  a 1 1 o n  s  w ere  repeated 


1-8 


f  0  r 

ten  sets 

o  f 

r  an doml  ■ 

1  formed 

three  v  ector 

s  for  each 

:.Pf 

a 

d l f f eren t 

UB 

Tj n  matrix  was 

genera  t  ed . 

The 

result 

E  O  f 

thl 

s  exercise 

ar  e 

shown 

in  rows 

of  2-10  of 

table 

I  . 

The 

total  sum  of  errors  corrected  in  each  column,  1 ,e.  for  each 
of  the  different  combinations  of  memory  masks  and  thresholding 
procedures,  are  shown  in  the  bottom  row  of  table  I .  The 
values  in  this  row  are  taken  as  a  performance  measure  of 
the  CAMs  simulated.  The  grev  level  case  with  zero  threshold 
is  taken  to  represent  a  relative  performance  of  100'!.  This 

type  of  CAM  was  able  to  correct  a  total  of  246  errors  total 

in  the  exercise.  However,  the  CAM  with  the  UB  mask  and  input 
energy  thresholding  with  relaxation  is  seen  have  slightly 
better  performance  bv  correcting  260  errors  i.e.  a  relative 

performance  of  105%.  Adaptive  input  energy  thresholding 

(without  relaxation)  is  seen  to  fare  well  at  7'?%  relative 

performance.  The  poorest  in  performance  was  the  CAM  with 
UB  mask  with  mean  of  projections  <proj.>  as  threshold  where 
the  relative  performance  is  seen  to  be  61%. 

4  .  EX  PERI  M  ENT  A  L  U  E  R.  I  F I  CAT  I  ON 


T o  v er i f  v  ex per i men  tall "  t he  per  f  o rman ce  o f  t he  ad ap 1 1 v e 
thresholding  schemes  of  Figs  8  and  9,  a  modified  version 
of  the  optical  implementation  described  in  section  2  is  utilized. 
In  this  modified  version  only  one  channel  of  the  two  arm 
system  of  Fig.  4  is  utilized.  The  binary  bipolar  mask  in 
this  arm,  which  represents  one  half  of  the  bipolar  binary 
T  matrix,  is  replaced  bv  a  unipolar  binary  version  of  the 
complete  T  matrix  consisting  of  32  rows  and  32  columns. 
The  32  columns  exactly  overlay  the  elements  of  the  32  element 
photo  detector  (F'D)  array.  For  the  results  reported  here 
the  PD  array  outputs  were  read  bv  an  A/D  converter  via  an 
analog  multiplexer  and  stored  in  a  DEC  11/23  Modular  INs t rumen  - 
tat  ion  Computer  as  illustrated  in  Fig.  10.  The  energy  of 


tns  input  vector  is  entered  in  the  computer  m an u a 1 1 1 1  a s  c n e 
half  the  number  of  element  a  in  the  input  LED  array  that  are 
lit.  The  adap  1 1 v e  t  hr  esho 1 d l n  g  al 90 r 1 t  hms  o f  Figs.  3  an d 
3  described  in  the  preceding  section  are  carried  out  then 
bv  t  he  comp  u  ter  wh  1  ch  f  urni  shes  32  v  o  1 1  age  v  alues  f  0  r  dr  1 v  1  n  9 
the  LED  array  for  subsequent  iteration.  For  the  adaptive 
thre s h 0 1 d 1 n g  s c h em e  of  Fig.  3 ,  these  v  alues ,  u h 1 c h  c 0 n s 1 s t 
of  an  array  of  zeros  and  ones  representing  the  result  of 
the  first  iteration,  are  easily  entered  in  the  system  manually 
through  the  word  composer.  For  the  adaptive  thresholding 
an d  r  el  ax  a  1 1 o n  scheme  t he  comp  u t  er  0 u tout  co n  s 1 s  t s  0  f  an 
array  of  numbers  each  ranging  between  zero  and  one  for  driving 
the  LEE1  array  through  an  appropriate  D/  A  c 0 n y  er  t  er  .  hs  this 
interf ace  was  n 0 t  comp 1 e  t  ed  at  the  1 1  me  0  f  t  h 1 s  wr 1  ting, 
results  for  the  adaptive  thresholding  (without  relaxation) 
are  included  here.  A  photograph  of  the  computer  generated 
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The  result  of  the  above  op  to -digital  experiment  for 
the  adaptive  thresholding  (without  relaxation  >  scheme  are 
presented  in  table  II .  These  show  that  the  CAM  with  UB  mask 
and  adaptwe  thresholding  functions  s  at  1  sf  actor  1 1"  and  is 
capable  of  recognizing  partial  inputs  of  the  mernorv  states. 
1  . e .  of  the  three  stored  vect o r s ,  up  to  5  errors  are  tolerated 
f  0 r  vector  1 ,  4  er r o r  s  f  o r  v ector  2  an  d  E  err  o  r  s  for  " e c  t  0 r 

3  for  both  normal  and  complement  of  vectors.  The  performance 
is  in  general  agreement  with  the  numerical  simulation  results 
presented  in  table  I  for  this  case.  Op  to -digital  experimentation 
of  the  type  described  above  is  found  in  our  wort-  to  be  a 
v  e  r  v  useful  step  in  the  stud  y  o  f  rn  0  d  e  1  s  of  neural  nets  fall:  n  g 


FIGURE  CAPTIONS 


Fig.  1  .  Co  n  cep  t  f  o  r  optical  i  mp  1  ernen  t  a  t  i  o  n  o  f  a  co  n  tent 

addressable  memory  based  on  the  Hopfield  model. 

(a)  matrix  vector  multiplier  incorporating  nonlinear 
electronic  feedback,  (b)  scheme  for  realizing  a 
binary  bipolar  memory  mask  transmittance  in  incoherent 
light. 

Fig.  2.  Stored  words,  their  Hamming  distance,  and  their 
clip  p  ed  T  memo  r  y  rna  t  r  i  x  . 

Fig.  3.  Two  halves  of  T  memory  mask. 

Fig.  4.  Arrangement  for  optical  implementation  of  the  Hoofield 
model,  (a)  Op  to -elect r on i c  circuit  diagram, 

( b )  Pictorial  u l ew . 

Fig.  5.  O' lews  of  (a)  input  LED  array  and  (to  memory  submask 
/  PD  array  assemblies. 

Fig.  G .  Wo r d  comp o ser  an d  d i sp 1  ay  bo x . 

F?g.  7.  Two  neurons  in  synaptic  contact  (a)  and  their 
op  t  o -el ec  t  r  on l c  an al o  g  ( b )  . 

Fig,  3 .  CAM  with  unipol ar  so n ap  1 1 c  mask  an d  ad ap  1 1 v e 

thresholding  scheme . 

Fig.  3.  CAM  with  unipolar  synaptic  mask  and  adaptive 
thresholding  with  relaxation. 

Fig.  10.  Experimental  opto -digital  arrangement  used  in  studying 
CAM  b e h a v i o  r  u  s i n  g  U B  mas k  a n  d  a d ap  t i v e  t  h r  e s h o 1 d i n g 
and  feedback  schemes. 

Fig.  11.  Un  i  p  o  1  ar  B  l  n  ar  v  T  memo  r  y  mask  , 

Table  I .  Comparison  of  different  tvpe  of  memory  mask  and 

thresholding  procedures  'digital  simulation'. 


Table  II.  Performance  of  the  optical  CAM  ( op to-di gi t al 


e  :x  per  i  m  e  n 


FEEDBACK, GAIN,  AND  THRESHOLDING 


(a) 


R 


Concept  for  optical  implementation  of  a  content  addressable 
memory  based  on  the  Hopfield  model.  (a)  matrix  vector  multi¬ 
plier  incorporating  nonlinear  electronic  feedback.  (b)  scheme 
for  realizing  a  binary  bipolar  memory  mask  transmittance  in 
incoherent  light. 
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Fig.  2.  Stored  words,  their  Hamming  distance,  and  their  clipped  Bipolar 
binary  (BB)  T  memory  matrix. 
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Arrangement  for  optical  implementation  of  the  Hopfield  model 
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ABSTRACT 

Optical  analogs  of  2-D  distribution  of  idealized  neurons  (2-D 
neural  net)  based  on  partitioning  of  the  resulting  4-D  connectivity 
matrix  are  discussed.  These  are  desirable  because  of  compatibility 
with  2-D  feature  spaces  and  ability  to  realize  denser  networks.  An 
example  of  their  use  with  sinogram  classifiers  derived  from  realis¬ 
tic  radar  data  of  scale  models  of  three  aerospace  objects  taken  as  learn¬ 
ing  set  is  given.  Super-resolved  recognition  from  partial  informa¬ 
tion  that  can  be  as  low  as  20%  of  the  sinogram  data  is  demonstrated 
together  with  a  capacity  for  error  correction  and  generalization. 

INTRODUCTION 

Neural  net  models  and  their  analogs  furnish  a  new  approach  to 
signal  processing  that  is  collective,  robust,  and  fault  tolerant. 

Optical  implementations  of  neural  nets^>2  are  attractive  because  of 
the  inherent  parallelism  and  massive  interconnection  capabilities 
provided  by  optics  and  because  of  emergent  optical  technologies  that 
promise  high  resolution  and  high  speed  programmable  spatial  light 
modulators  (SLMs)  and  arrays  of  optical  bistability  devices  (optical 
decision  making  elements)  that  can  facilitate  the  implementation  and 
study  of  large  networks.  Optical  implementation  of  a  one-dimension¬ 
al  network  of  32  neurons  exhibiting  robust  content-addressability 
and  associative  recall  has  already  been  demonstrated  to  illustrate 
the  above  advantages. 3  Extension  to  two-dimensional  arrangements 
are  of  interest  because  these  are  suitable  for  processing  of  2-D 
image  data  or  image  classifiers  directly  and  offer  a  way  for  optical 
implementation  of  large  networks.^ 

In  this  paper  we  will  discuss  content  addressable  memorv  (CAM) 
architectures  based  on  partitioning  of  the  four  dimensional  T  ^ 

memory  or  interconnection  matrix  encountered  in  the  storage  of  2-D 
entities.  A  specific  architecture  and  implementation  based  on  the 
use  of  partitioned  unipolar  binary  (u.b.)  memory  matrix  and  the  use 
of  adaptive  thresholding  in  the  feedback  loop  are  described.  The 
use  of  u.b.  memory  masks  greatly  simplifies  optical  implementations 

3  4 

and  facilitates  the  realization  of  larger  networks  -(10  -10  neurons). 
Numerical  simulations  showing  the  use  of  such  2-D  networks  in  the 
recognition  of  dilute  point-like  objects  that  arise  in  radar  and 
other  similar  remote  sensing  imaging  applications  are  described. 

Dilute  objects  pose  a  problem  for  CAM  storage  because  of  the  small 


IT-1 


Hamming  distance  between  them.  Here  we  show  that  coding  in  the  form 
of  a  i'inog’uzm  ctcu>6-iQ-LZA.  of  the  dilute  object  can  remove  this  limita¬ 
tion  permitting  recognition  from  partial  versions  of  the  stored 
entities.  The  advantage  of  this  capability  in  super-resolved  recog¬ 
nition  of  radar  targets  is  discussed  in  the  context  of  a  new  type  of 
radar  diversity  imaging,  studied  extensively  in  our  laboratory,  that 
is  capable  of  providing  sinogram  information  compatible  with  2-D  CAM 
storage  and  interrogation.  Super-resolved  automated  recognition  of 
scale  models  of  three  aero-space  objects  from  partial  information  as 
low  as  20%  of  a  learned  entity  is  shown  employing  hetero-associative 
storage  where  the  outcome  is  a  word  label  describing  the  recog¬ 
nized  object.  Capacity  for  error  correction  and  generalization  were 
also  observed. 

TWO-DIMENSIONAL  NEURAL  NETS 

Storage  and  readout  of  2-D  entities  in  a  content  addressable  or 
associative  memory  is  described  next.  Given  a  set  of  M  2-D  bipolar 


binary  patterns  or  entities  v^1^  m=l,2. 


.M  each  of  NxN  elements  rep¬ 


resented  by  a  matrix  of  rank  N,  these  can  be  stored  in  a  manner  that 
is  a  direct  extension  of  the  1-D  case  as  follows:  For  each  element  of 
a  matrix  a  new  NxN  matrix  is  formed  by  multiplying  the  value  of  the 
element  by  all  elements  of  the  matrix  including  itself  taking  the 
self  product  as  zero.  The  outcome  is  a  new  set  of  N2  binary  bipolar 
matrices  each  of  rank  N.  A  formal  description  of  this  operation  is, 


T(">  -  {"if 

ijkil  Q  3 


i=k,  j  =  l 


which  is  a  four  dimensional  matrix.  An  overall  or  composite  synaptic 
or  connectivity  memory  matrix  is  formed  then  by  adding  all  i-D 

matrices  T^??,  i.e., 
ljkx, 

T.  ,  =  I  T(m?  .  (2) 


This  symmetric  4-D  matrix  has  elements  that  if  :n  ••  1 1  uo  a  -t  ween  -M 
to  M  also  in  steps  of  two  as  for  the  L-D  neur  li  net  i  .e  mo  winch 
assume  values  of  +1  and  -1  (and  zeros  for  the  self  product  elements) 
when  the  matrix  is  clipped  or  binarized  as  is  usuailv  referable  for 
optical  implementations.  Two  dimensional  unipol  r  binarv  entities 

b^^are  frequently  of  practical  importance.  These  can  be  transformed 
in  the  usual  way  into  bipolar  binarv  matrices  through  v *m 1 = < 2b . ™ ^ -1 ) 

lj  i] 

which  are  then  used  to  form  the  4-D  connectivity  matrix  or  memory  as 
described.  Also,  as  in  the  1-D  neural  net  case,  the  prompting  entity 

can  be  unipolar  binary  b^,  which  would  simplify  further  optical 


implementations  in  incoherent  light. 

Architectures  for  optical  implementation  of  2-D  neural  nets 
must  contend  with  the  task  of  realizing  a  4-D  memory  matrix.  Here 
a  scheme  is  presented  that  is  based  on  the  partitioning  of  the  4-D 
memory  matrix  into  an  array  of  2-D  matrices  of  rank  N. 

Nearest  neighbor  search  of  the  memory  matrix  for  a  given  entity 

bij*°^  c*one  ^  forming  the  estimate. 


r (mo)  _  N  ,  (me 

ij  ’  ijk£  k£ 


....  i, j , k, &  =  1,2, ...N 


followed  by  thresholding  to  obtain  a  new  u.b.  matrix  which  is  used 
to  replace  b^°^  in  eq.  (3)  and  the  procedure  is  repeated  until  the 
resulting  matrix  converges  to  the  stored  entity  closest  to  the  ini¬ 
tiating  matrix  b^™°^ .  The  operation  in  eq.  (3)  can  be  interpreted 
as  first  partitioning  of  the  4-D  ^  matrix  into  an  array  of  2-D 

submacrlces  of  rank  Si  Ti2kjT  ' ' '  ’  T1NU'  T21k£’  T22U . 

T2Nkr  ••••  TSlkr  W .  TNNU  as  dePi<:ted  srkematically  In  Fig. 

1(a)  where  the  partition  submatrices  are  arranged  in  a  2-D  arrav. 


This  first  step  is  followed  by  multiplication  of  b. 


by  each  of  the 


partition  submatrices,  on  an  element  by  element  basis,  and  summing 

the  products  for  each  submatrix  to  obtain  the  first  estimate  b^™0"1. 

The  tensor  multiplications  and  summation  operations  called  for  in 
eq.  (3)  are  carried  out  in  Fig.  1(a)  by  placing  a  spatially  inte¬ 
grating  photodetector  (PD)  behind  each  submatrix  of  the  partitioned 
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Fig.  1.  Optical  analog  of  2-D  neural  net.  (a)  Architecture  based 
on  partitioning  of  connectivity  matrix,  (b)  Opto-elec- 
tronic  embodiment. 


memory  mask  which  is  assumed  for  the  time  being  to  be  realized  by 
pixel  transmittance  modulation  in  an  ideal  transparency  capable  of 

assuming  negative  transmittance  values.  The  input  entity  b^0^  is 
assumed  to  be  displayed  on  a  suitable  LED  array.  The  LED  display  of 
b^™°^  is  multiplied  by  the  ideal  transmittance  of  each  of  the  parti¬ 
tion  submatrices  by  imaging  the  display  on  each  of  these  with  exact 
registration  of  pixels  by  means  of  a  lenslet  array  as  depicted  in 
Fig.  1(b).  The  output  of  each  PD,  proportional  to  one  of  the  com¬ 
ponents  of  eq.  3,  is  thresholded,  amplified,  and  fed  back  to  drive 
an  associated  LED.  The  (i,j)-th  LED  is  paired  with  the  (i,j)-th  PD. 
This  completes  the  interconnection  of  the  2-D  array  of  NxN  neurons 
in  the  above  architecture  where  each  neuron  communicates  its  state 
to  all  other  neurons  through  a  prescribed  four  dimensional  synaptic 
or  memory  matrix  in  which  information  about  M  2-D  binary  matrices  of 
rank  N  (entities)  have  been  stored  distributively .  The  number  of 
2-D  entities  that  can  be  stored  in  this  fashion  is  M  =  N2/8£nN,  which 
follows  directly  from  the  storage  capacity  formula  for  the  1-D  neural 
net  case  by  replacing  N  by  N^. 

The  added  complexity  associated  with  having  to  realize  a  bi¬ 
polar  transmittance  in  the  partitioned  T^^jj  memory  mask  of  Fig.  1 

can  be  avoided  by  using  unipolar  transmittance.  This  can  lead  how¬ 
ever  to  some  degradation  in  performance.  A  systematic  numerical 
simulation  study^  of  a  neural  net  CAM  in  which  statistical  evalua¬ 
tion  of  the  performance  of  the  CAM  for  various  types  of  memory  masks 
(multivalued,  clipped  ternary,  clipped  u.b.)  and  thresholding  schemes 
(zero  threshold,  adaptive  threshold  where  energy  of  input  vector  is 
used  as  threshold,  adaptive  thresholding  and  relaxation)  was  carried 
out.  The  results  indicate  that  a  u.b.  memory  mask  can  be  used  with 
virtually  no  sacrifice  in  CAM  performance  when  the  adaptive  thresh¬ 
olding  and  relaxation  scheme  is  applied.  The  scheme  assumes  an 
adaptive  threshold  is  used  that  is  proportional  to  the  energy  (total 
light  intensity)  of  the  input  entity  displayed  by  the  LED  array  at 
any  time.  In  the  scheme  of  Fig.  1(b)  this  can  be  realized  by  pro¬ 
jecting  an  image  of  the  input  pattern  directly  onto  an  additional  PD 
element.  The  PD  output  being  proportional  to  the  total  intensity  of 
the  input  display  is  used  as  a  variable  or  adaptive  threshold  in  a 
comparator  against  which  the  outputs  of  the  PD  elements  positioned 
behind  the  partitioned  components  of  the  T„^jj  mernory  mask  are  com¬ 
pared.  The  outcomes,  now  bipolar,  are  attenuated  and  each  is  fed 
into  a  limiting  amplifier  with  delayed  feedback  (relaxation) .  Each 
] imiter/amplif ier  output  is  used  to  drive  the  LED  that  each  photo¬ 
detector  is  paired  with.  It  was  found^  that  this  scheme  yields  per¬ 
formance  equivalent  to  that  of  an  ideal  CAM  with  multivalued  connec¬ 
tivity  matrix  and  zero  thresholding.  Note  that  although  the  ini- 

tializaing  2-D  entity  b^?0^  is  unipolar  binary,  the  entities  fed 

back  after  adaptive  thresholding  and  limited  amplification  to  drive 
the  LED  array  would  initially  be  analog  resulting  in  multivalued 
iterates  and  intensity  displays.  However,  after  few  iterations  the 


outputs  become  binary  assuming  the  extreme  values  of  the  limiter.  The 
ability  to  use  u.b.  memory  matrices  in  the  fashion  described  means 
that  simple  black  and  white  photographic  transparencies  or  binary 
SLMs  can  be  used  respectively  as  stationary  or  programmable  synaptic 
connectivity  masks  as  suggested  -by  Fig.  1. 

SINOGRAM  CLASSIFIERS  AND  HETEROASSOCIATIVE  STORAGE 

Sinograms  are  object  representations  encountered  in  tomography^1  ’  ^ 
They  are  also  useful  as  object  classifiers  specially  when  the  objects 
are  point-like  and  dilute^.  Civen  a  set  of  2-D  dilute  objects  the 
Hamming  distances  between  their  sinogram  classifiers  will  be  greater 
than  the  Hamming  distances  between  the  objects  themselves,  with  both 
sets  digitized  to  the  same  number  of  pixels,  making  it  easier  for  an 
associative  memory  to  distinguish  between  the  sinograms^.  Sinogram 
classifiers  have  additional  advantages  that  enable  scale,  rotation, 
and  shift  invariant  recognition  of  radar  target  which  can  not  be 
detailed  here  because  of  limited  space.  A  sinogram  is  a  cartesian 
plot  of  the  polar  projections  of  object  detail.  For  example  referring 
to  Fig.  2(a)  which  represents  a  dilute  object  consisting  of  16  points 
on  a  32x32  pixel  grid,  the  distance  that  the  projection  of  each  point 
makes  on  the  y  axis  as  measured  from  the  origin  when  the  object  is 
rotated  about  the  origin  traces  a  sinusoidal  pattern  when  plotted 
against  rotation  angle  as  shown  in  Fig.  2(b).  Figure  2(c)  is  a 


Fig.  2.  Sinogram  generation,  (a)  Sparse  object,  (b)  Sinogram,  (c) 

Digitized  sinogram,  (d)  Experimental  sinogram  generation  in 
radar  by  range-profile  measurement. 

digitized  version  of  the  sinogram  of  Fig.  2(b)  plotted  on  a  32x32 
pixel  grid.  The  sinogram  of  a  radar  target  is  produced  by  measuring 
the  differential  range  or  range-profile  of  the  target  employing  the 
arrangement  of  Fig,  2(d).  The  system  basically  measures,  with  high 
resolution,  the  differential  distance  (differential  range  or  range- 
profile)  from  the  rotation  center  of  the  projections  of  the  scatter¬ 
ing  centers  of  the  object  (here  scale  models  of  aerospace  targets) 
on  the  line-of-sight  or  the  radar  system.  Cartesian  plots  of  the 
differential  distance  of  range-profile  versus  azimuthal  angle  of 
rotation  <J>  results  in  a  sinogram  classifier  or  feature  space  of  the 
target  which  characterizes  it  at  any  fixed  elevation  angle  -.  The 
top  row  of  Fig.  3  shows  three  digitized  sinogram  classifiers  of 
scale  models  of  three  aerospace  targets  plotted  on  a  32x32  pixel 


grid.  These  are  treated  as  a  learning  set  and  stored  hetero-associa- 
tively  rather  than  autoassociatively  be  replacing  v in  eq.  (1)  by 


(m)  (m) 

rk£  k»2'=1*2--  .32;  m=l,2,3  where  r^  represents  abbreviated  word 

labels  shown  in  the  bottom  row  of  Fig.  3  with  which  the  three  test 
objects  are  to  be  associated. 
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Fig.  3.  Hetero-associative  storage.  Digitized  sinograms  (top)  and 
associated  word  labels  (bottom). 

RESULTS 

Representative  results  of  numerical  simulation  of  exercising  the 
heteroassociative  memory  matrix  with  complete  and  partial  versions  of 
one  of  the  stored  entities  in  which  the  fraction  n  of  correct  bits  or 
pixels  in  the  partial  versions  ranged  between  1  and  .1  are  presented 
in  Fig.  4.  Reliable  recognition  was  found  to  occur  following  one  itera¬ 
tion  for  all  entities  stored  down  to  n  =  .2.  For  0  =  .1  or  less  success¬ 
ful  recall  of  correct  labels  was  found  to  depend  on  the  angular  location 
of  the  partial  data  the  memory  is  presented  with  as  illustrated  in  the  two 
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Fig.  4.  Example  of  recognition  from  partial  information.  Complete 

and  partial  sinograms  of  data  set  3  used  as  input  (top),  and 
final  memory  state-recognized  label  (bottom). 
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right-most  examples  in  Fig.  4.  Here  the  memory  could  not  label  the 
partial  input  correctly  but  converged  instead  onto  a  label  that  it 
did  not  learn  before.  This  appears  to  be  a  generalization  (mixture) 
of  the  three  entities  stored  earlier.  This  is  quite  analogous  to  the 
generalization  capability  of  the  brain.  Note  the  generalization  is 
contrast  reversed  as  we  know  that  stable  states  of  a  memory  with 
symmetric  connectivity  matrix  are  not  only  the  entities  stored  but 
also  their  compliments. 

CONCLUSIONS 

Architectures  for  optical  implementation  of  2-D  neural  nets 
based  on  partitioning  of  the  4-D  connectivity  matrix  are  shown  to  be 
suitable  for  use  with  2-D  object  classifiers  or  feature  spaces.  An 
example  of  their  utility  in  super-resolved  recognition  (labeling)  of 
radar  targets  characterized  by  sinogram  classifiers  is  presented. 

The  results  show  that  neural  net  models  and  their  opto-electronic 
analogs  furnish  a  new  viable  approach  to  signal  processing  that  is 
both  robust  and  fault  tolerant. 
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APPENDIX  III 


Phased  Array  Antenna  Pattern  Synthesis 

By  Simulated  Anneal  ins 

N.  Farhat  and  B.  Bai 
University  of  Pennsylvania 
The  Moore  School  of  Electrical  Ensineeina 
Electro-Optics  And  Microwave  Optics  Laboratory 
Philadelphia-  PA  19104 

ABSTRACT!  A  new  procedure  is  described  for  optimum  phased  array 

synthesis.  The  synthesis  is  optimized  by  a  simulated  anneal  ins 

-  o  c  e  s  s  in  which  the  energy  function  is  oirectl  y  related  to  the  far 

'  i  e  1  :  intensity  a  Phased  array.  Numerical  simulation  results  are 

presented.  A  pqssis  le  optical  - digital  hono  implementation  t  h  a  t  can 
perform  the  required  computation  at  hisher  speed  than  a  pure  digital 
implementation  is  discussed. 

1 . INTRODUCTION 

"  ’3  r  the  s  y  nthesis  of  an  antenna  array  with  u  mfo  r  m  1  v  s  p  a  o  e  o 

:  :  ■;  0  1  1  h  n  o  -in  that*  l  *  the  c  urr  ?  r  t  d  l  s  t  r  i  i  u  t  i  o  n  -  ur  c  t  :  o  •*. 

•  5  not  r  e  5  t  r  l  c  t  e  d  *  a  Dolph  —  Cheb-/  she1*  distribution  function  over  t  *  e 
antenna  aives  rise  to  an  optimum  pattern  which  has  the  lowest  sidelooe 

-  e  ■ )  e  .  '  o  "  a  specified  m  a  l  n  o  e  a  m  wiathCll.  H  o  w  e 1  >  e  r  •  i(  c  o  t  the  a  u  r  p  o  s  e 

-  -  i3  =  '  3  '  a  c  t  i  o  a  1  implementation,  t  n  e  d  l  s  t  i  o  u  1 1  o  n  *  o  1 1  o  n  i  ; 

-  0  s  t  r  1  0  t  e  d  to  some  specific  set-  other  methods  h  a 1  *  e  to  oe  i  1 1  sst 

and  used  for  optimum  synthesis.  The  simulated  annealing  method 
presented  here  is-  by  our  study.  one  of  the  choices  for  optimum 
s/nthesis  of  phased  arrays  with  restricted  distribution  functions. 
The  synthesis  is  optimal  in  the  sense  that  the  lowest  sidelobe  level 
is  achieved  while  the  specified  mainbeam  width  is  maintained.  This 
method  can  be  used  for  both  microwave  phased  arrays  and  optical 


arrays.  In  our  study  so  far.  we  have  been  mainly  concerned  with 
optical  array Sr  which  appear  to  be  technolosically  feasible  with  the 
present  electronic  and  optical  t e chn o  1  03 i es .  Hence  >  the  parameters 
assumed  in  the  simulations  below  are  relevant  to  the  optical  case>  but 
the  method  and  conclusion  apply  to  phased  arrays  in  general . 

2. SIMULATED  ANNEALING  METHOD 

Metropolis  et  a  1 .  introduced  the  simulated  anneal ing  alsorithm  for 
calculating  the  properties  of  any  substance  of  interacting  1 n  d 1 » 1 d u a  1 


mo  1 e c u 1 e s C 2 I .  The  alsorithm  was  Previously  applied  to  some 
optimization  problems  >  including  physical  design  of  c  0  m  p  u t  e  r  s  r  and  the 
traveling  salesmn  problemC31.  The  method  can  be  extended  for  general 
optimization  problems.  For  the  system  to  be  optimizedr  an  "energy" 
function  E  is  first  established  and  a  dynamic  variable  T,  the 
"temperature"  of  the  system-  is  chosen  to  control  the  process. 
Starting  at  a  h 1 s h  “temperature" »  the  system  is  slowly  cooled  do w r > 
until  the  system  "freezes"  and  reaches  the  optimum  state  in  a  manner 
similar  to  annealing  of  a  crystal  during  growth  to  reach  a  near 


»erf ec t 

structure 

.  At  each 

“temperature" 

r  3 

change  in  the  s 

v  ste  n\ 

•"  a  0  3  a  c 

c  0  r a  1 n  3  to 

a  certain  ru 

1 e -  and  then 

the 

"  9 n e  r 3  y  "  c  h an 3  e 

s  y  s  t  3  m 

A  2  15  calc 

u  1  a  t  e  d .  I" 

»  tie  s  y 

s  t  3  m 

a  1 t  9  r  a  t ; 0 n  is 

n  9  v  3  1 

and  the 

process  is 

continued. 

The  acceptance 

or  rejection 

0  f 

alteration  or  change  of  grain  of  the  system  when  A E > 0  is  treated 
probabilistically.  Accordingly!  the  Boltzman  factor  f (AE  )  =  e xp < -£E/KT ) 
is  calculated-  where  K  is  a  constant  whose  dimension  depends  on  the 
dimensions  of  AE  and  T.  Then  a  random  number  R  uniformly  distributed 


in  the  inter uial  CO-1)  is  chosen 


If  R<f(&E>-  the  change  of  grain  is 


retained;  on  the  other  hand  t  if  R>f(^E)r  the  change  is  discarded . 
that  is.  the  system  before  chanse  is  used  for  the  next  step  of  the 
process.  This  procedure  is  repeated  for  each  "temperature"  until  the 
system  is  optimized  to  reach  a  global  energy  minimum.  The  choices  of 
K  and  the  initial  T  are  crucial  for  the  success  and  speed  of 
conoergence  of  simulated  annealing  process.  Because  of  the 
probabilistic  Boltzman  selection  rule  of  the  AE>0  case.  the  process 
can  always  get  out  of  a  local  minimum  of  the  "energy"  function  in 
which  it  could  get  trapped  and  proceed  to  the  desired  global  minimum, 
’hi ;  maKes  simulated  annealing  different  from  iterating  improvement 
pracedure[31-C6] . 


3. ANALYTICAL  CONSIDERATIONS 

For  a  2-dimensional  phased  array  with  (2M+1)  (2N+1)  identical 

subapertures ( elements  )  .  the  array  function  can  be  expressed  as[73  > 


m  i \  =  -fi 


0) 


where.  A  is  the  .pacin 
lj  direction!  is 

o  3  o  a  s  e  mo  d  ulatio  n 
suoanert  '.i  re  f  u  n  c  1 1  o  n 

symbol  '  *  '  stands  for 


g  between  elements  in  ^  direction  and  B  in 
a  complex  quantity  and  represents  the  amplitude 
o*'  the  (m.n)th  sub  aperture;  t  (  £  .lj  ;  is  the 
and  is  the  same  for  all  the  s  u  b  a  p  e  r  t  u  r  e  s  ‘  the 
convolution. 


The  scalarized  far  field  pattern  G(fx.fy)  of  a  Phased  array  is 
proportional  to  the  Fourier  transform  (Fraunhofer  pattern)  of  its 
array  function  g  C 7 3 .  Therefore,  if  the  proportionality  constant 


is  ignored,  the  far  field  can  be  expressed  as 


Since  T(fx.fy)  is  fixed  for  the  aiuen  subaperture  function  and  it 
names  much  more  slowly  than  the  array  factorr  the  effect  of  the 
subaperture  factor  is  i n s i sn i f i can t  in  the  present  synthesis.  The 
array  factor  A(fxrfy)  will  be  studied  by  chansina  the  weiahtina  factor 
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for  all  possible  m  and  n  to  achieve  the  optimum  radiation  pattern 


The  "enera/"  function  in  simualted  annealins  can  be  established  in 
many  different  ways  for  phased  array  synthesis.  Since  our  primary 
wori;  is  done  for  optical  and  infrared  Phased  arrays  and  this  Kind  of 
arrays  has  a  relatively  large  size  compared  to  the  wauelenath  X.  used 
(X  on  th  order  of  10”^  m)*  the  beam  width  is  very  small  (on  the  order 
o f  1  1T'2  dearees).  It  is  important  to  achieve  the  lowest  sidelobe 
level.  For  this  purpose*  one  obvious  way  to  choose  the  "energy" 

''unction  for  Phased  array  synthesis  would  be  the  energy  outside  a 
specified  main  lobe.  When  this  energy  function  is  minimized  *  if  the 
total  energy  remains  constant*  it  could  be  expected  that  the  energy 
would  become  concentrated  in  the  main  beam.  Consesuente  1  •/  *  the 
relative  sidelobe  leuel  could  be  minimized.  But*  from  simulations 
that  we  have  run*  it  turns  out  that  this  energy  function  cannot 
minimize  the  sidelobe  leuel.  This  is  understandable*  if  one  recalls 
tie  pattern  given  3*/  the  Do  i  p  h-Ch  e  D  y  s  h  e  v  distribution*  that  instead  of 
concentrating  in  the  main  beam*  the  energy  has  the  tendency  to 
distribute  itself  equally  among  all  the  side  lobes  as  the  sidelobe 
e  <  >  e  1  is  optimized.  This  does  not  mean  that  the  energy  in  the  side 
'dies  : 5  m l n i m i z e d .  In  the  simulation  presented  here*  toe  i n t  e n s l t y  Z 
associated  with  the  highest  sidelobe  leuel  is  used  as  the  "energy" 
function  for  simulated  annealing.  Of  course*  if  necessary*  a 
constraint  about  the  beamwidth  could  be  included  in  the  "energy" 
function.  When  the  "energy"  is  minimized*  the  relative  sidelobe  leuel 
is  now  also  minimized.  The  beamwidth  at  the  final  run  will  be  taken 
as  the  "specified"  beamwidth  with  the  final  weights.  The 


relationship  of  beamwidth  and  sidelobe  ley el  is  then  optimum  in  the 
sense  of  Dolph-Chebyshev*  that  i;>  the  sidelobe  level  is  the  lowest 
for  the  si  wen  beamwidth. 

4 .SIMULATION 

Since  the  simulated  annealing  process  is  the  same  for  a  2-dimensional 
array  as  for  a  1- dimensional  array  and  considering  the  limited 

computation  power  of  the  MICRO  PDP-11  computer  available  in  our 

simulation*  a  l-dunensional  array  is  used  1 n  our  study.  T  h e 
1- dimensional  array  is  also  assumed  to  be  a  continuous  one  with  m  a  n  y 
desired  Pixels.  Each  of  the  pixels  acts  as  a  suoaperture.  The 

simulation  starts  with  a  uniform  distribution  (all  subapertures  with  1 
or  -1).  The  far  field  pattern  is  shown  in  Fis.2  (a)  for  this  uniform 
distribution  case.  The  distribution  function  is  restricted  to  the  set 
of  1>  -lr  which  means  real  transmittance  with  binary  phase  ( p  h  a  s  e  =  0  or 
’  modulation.  Then*  simulated  annealing  is  carried  o  u  *  s  v  j  u  s  t 
c n a  n a i n a  the  s i a  n  of  each  subanerture  in  t  u r n  *  calculating  t  h e 

intensity  of  the  highest  sidelobe  and  a p •»  1  y l n 3  the  algorithm  for  each 
change.  The  final  optimum  result  is  given  in  Fia.2  (b).  This 
simulation  is  done  for  an  array  o ?  4 1  subapertures.  ~ h e  subaperture 
sice  is  assumed  to  n?  e«uai  to  t«e  spacing  A  bet-ieen  suoapertuers  and 
is  taken  to  be  6lA (figure  relevant  for  optical  arrays).  The  element 
distribution  function  that  aives  the  final  optimum  result  is  shown  in 
table  1.  This  function  aives  optimum  pattern  only  for  broad  side 
direction  (0=0).  LiKe  the  Dolph-Chebyshev  optimum  function*  the 
optimum  function  given  by  simulated  annelina  is  different  for 
different  steering  anales*  but  an  optimal  weiaht  can  be  computed  for 


each  steerina  an  ale.  Front  the  simulation  results  it  is  seen  that  the 
far  field  pattern  has  similar  features  to  the  pattern  aiuen  b  /  the 
Dolph-Cheb'/sheu  distribution  function.  In  the  Dolph-Chebvshev 

pattern/  all  the  side  lobes  have  the  same  leu el  for  a  specified 
beamwidth.  A  numerical  example  in  [1]  shows  an  8-element  array 
(element  separation  d  =  0.5A  )  with  25.8-dB  sidelobe  1  e  u  e  1  and 
•A  cj .  s  beamwdth.  The  optimum  pattern  aiuen  b  •/  our  simulation  shows 
nearly  equal  leu el  side  lobes  which  are  minimized  for  the  aiuen 
0 e  amw i d t  h . 

'5.  DISCUSSION 

Simulated  annealina  is  a  modification  of  the  iterative  improvement 
alaorithmCA].  It  is  physically  more  meaninaful  and  can  be  computed 
more  systematically  than  the  iterative  improvement [43  .  Physically/ 
simulated  annealina  process  is  analoaous  to  the  coolina  of  atoms  in 
■z  '  ■  •  s  t  a  1  3  r  o  w  t  !  c  a  •"  e  ^  u  1  annealina  produces  a  cef  ec  t-* -ee  crystal- 
-  a  p  :  a  anneal:  n  a  ~  c  o  u  c  e  s  a  defective  crystal  or  slass  [31  .  “  1  e 

probabilistic  treatment  with  the  probability  function 

P <&£) =exo ( -^£/KT )  provides  a  way  to  accept  the  unfavorable  chanaes  and 
■;  much  easier  to  compute.  Prom  our  simulation/  it  i a s  been  Pound 
t  ">  a  t  t  n  e  s  :  •>  u  1  a  t  e  c  anneal  m3  alsoritmi  seems  a  1  wa  s  to  a  i  *•  e  better 
performance  than  the  iterative  improvement  alaonthm. 

Since  simulated  annealina  is  a  modified  iterative  improvement 
process/  it  taKes  a  relatively  Iona  time  to  do  an  optimization  problem 
just  as  iterative  improvement  does  in  a  computer  calculation.  The 
phased  array  synthesis  in  our  simulation  runs  for  one  hour  or  so  for 


an  array  of  41  elements  on  a  MICRO  PDP-11  computer.  Findins  an 
efficient  scheme  to  reduce  the  excessive  amounts  of  computer  time  on 
most  optimum  problems  has  always  been  of  concernC5]C8].  Otherwise,  if 
enouah  computation  power  is  available  iterative  improvement  can  be  run 
from  random  starts  for  man y  times  to  approach  the  optimum  state.  Fast 
opto-disital  computins  schemes  similar  to  those  described  inC9I  may 
also  be  considered  for  phased  array  synthesis  by  simulated  anneal  ins. 
It  is  understood  that  the  far  fi.-ld  is  the  Fourier  transform  of  the 
a  r  a  y  distribution  function.  An  optical  lens  can  be  used  for 

com=*utina  the  Fourier  transform  as  the  distribution  function  is 
inputed  to  the  front  focal  plane  of  the  lens  via.  for  example.  an 
appropriate  compuer  driven  spatial  liaht  modulator  (SLM).  The  Fourier 
transform  in  the  bacK  focal  plane  can  be  recorded  and  fed  to  the 
computer/controller  to  maKe  the  simualted  annealins  decision.  The 
outcome  is  fedbacK  to  the  SLM  to  chanse  the  distribution  function  in 
the  front  focal  plane.  This  °  r  o  c  e  s  s  can  be  repeated  for  every  step  in 
simulated  anneal  in a.  The  hybrid  opto-diaital  scheme  will  oo  tie 
Fourier  transform  instantly.  In  this  fashion  the  computation  time 
associated  with  the  Fourier  transform  can  be  virtually  eliminated 
a  s  s  u  m 1 n  a  a  lush  speed  SLM  and  computer  interface  are  utilized.  ^  n 
«  t  o-e  ’  e  c  t  r  on  i  c  Boltzman  machine  *  or  accel  erat  l  na  tne  selection  r  u  2  e 
has  also  been  proposed  earlier  in  C  9  ]  .  Also,  a  Cauchy  probability 
selection  rule,  instead  of  the  Boltzman  selection  rule,  can  be  used  to 
speed  up  the  whole  annealina  process  furtherC83.  As  claimed  in  C8  3. 
the  computational  savins  of  simulated  annealina  with  Cauchy 
probability  selection  when  comparied  with  the  one  with  Boltzman 
selection  is  similar  to  that  FFT  comparied  with  DFT. 
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Simulated  Annealing  Synthesis 


(a)  mainbeam  width:  4 . 58E-2 (degree) 
view  range:  [-0.004,  0.004] 


(b)  mainbeam  width:5.5E-2  (degree) 
view  range : [ -0 . 004 , 0 . 004 ] 

#  of  elements:  2*20+1=41 


Fig.  2.  simulated  annealing  result  (a)  for  the  uniform  distribution 
(b)  the  simulated  annealing  result 


